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ABSTRACT 

The assumption behind this paper is that teachers 
should adopt the strategy of simplification of a learning task by 
integration that captures the inherent tendency for the integration 
of the objective world into subjective wholes, rather than through 
simplification by isolation of topics. An effective way of 
establishing the theme of a course is to use thematic organizers to 
portray a holistic picture of the entire domain of the course. This 
thematic approach was adapted for a course in tests and measurement 
taught in the school of education of a university. Two thematic 
organizers were produced, one for the entire course and one for the 
section on classroom statistics. The construction of thematic 
organizers is a continuous process, subject to revision because of 
improvement in the instructor's understanding of material and changes 
in course requirements. The guiding principle was that without 
knowing the theme, students could not organize their 
domain-comprehensive knowledge on the way to domain-specific 
professional knowledge. Formal evaluation has not been made of this 
application of the whole-theme approach, but informal observation 
indicates that students have maintained positive attitudes and have 
benefited from the approach. (Contains 23 references.) (SLD) 
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A Thematic Approach to Teaching Tests and Measurement 
A widespread assumption in education is that naive students of a course come to class with a 
head empty of the subject matter knowledge. Accordingly, they must receive and store the basic-level 
knowledge teachers present to them. This assumption is so seductive that it continues to be practiced 
beyond introductory courses, even after the students are well into their advanced graduate courses. 
Two detrimental consequences follow. First, the students' intuitive knowledge about the subject 
matter is completely ignored. Learning thus becomes entirely dependent on external sources, leaving 
many of the learner's own potential contributions to learning unutilized. Secondly, it encourages the 
teacher to adopt a piecemeal approach where the complex body of the knowledge of a domain is 
presented in a fragmented fashion. 

The fragmentation is based on the belief that learning can be made easier if the teacher breaks 
down the subject matter and teaches it one chunk at a time (Iran-Nejad, Hidi, & Wittrock, 1992). To 
many, chunks or pieces are basic elements for building the more complex knowledge of the domain; 
students can learn the basic concepts and principles now and put them together later. The problem 
with this simplification by isolation is that the teacher can be, and often is, caught up in the process 
of presenting individual pieces and fail to provide the learner with a way of bringing the various 
pieces together. Thus, students never get any picture of the entire domain of knowledge they are to 
learn or any clear ideas about the connections among various pieces. This explains why students 
always finish a course wondering how much of the knowledge of the domain they have acquired and 
how they can use the knowledge they have learned. It may also explain why much of what they have 
acquired is inert knowledge (Whitehead, 1929, Bransford, Franks, Vye & Sherwood, 1989) that does 
not apply to situations outside the classroom. 

Simplification by isolation has dominated both psychological and educational arenas for a long 
time. We can find it in Ebbinghaus* experimental study of nonsense syllables and in much of the 
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laboratory work in information processing research (e.g., Schneider & Shiffrin, 1977). In education, 
the assumption of simplification by isolation is responsible for the hierarchical structure of educational 
taxonomies which dominate the classroom teaching practices in American schools today (Bloom, 
Englenhart, Furst, Hill & Krathwohl, 1956, Quellmalz, 1985). The assumption is so deeply 
entrenched in the traditional school curriculum that few people stop to question its validity or consider 
possible alternatives. 

Tests and Measurement, for example, is a required course for undergraduate teacher 
education majors in our university. We include in the course such topics as objectives, assessment 
methods, test construction, classroom statistics, and grading. The course is usually taught one 
isolated topic at a time. It is left to the students (a) to make sense of the course material, (b) to figure 
out why certain topics (as opposed to others) are included in the course, (c) to think about how 
individual topics are related to one another, and (d) to grasp the entire picture of the domain of Tests 
and Measurement while working on individual topics. In most cases the course material is presented 
in a sequential way. Each individual topic is dealt with systematically. However, little attempt is 
built into the course mechanics to help students construct and maintain the entire picture of the whole 
domain. Thus, students blindly follow the instructor and do what they are told. The question is: 
without knowing the overall picture of the entire domain, how confident can students be about what 
they have learned about the subject? 

There is surprisingly little research on the topic of teaching Tests and Measurement. The 
few empirical articles we found dealt with assessment literacy among preservice and inservice teachers 
(Vanleirsburg & Jones, 1991, Jones & Davis, 1991) or provided teachers with the basic training in 
Testing and Measurement theory and practice. Invariably, the teaching approach adopted in these has 
been the conventional piecemeal approach. 
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The Whole-Theme Approach 
Simplification by isolation is not the only way teachers can simplify the learning task for their 
students. Gestalt psychologist suggested that building wholes, rather than breaking them, was the 
more natural simplifying process capitalizing on the inherent tendency of the human perception to 
impose coherence, regularity, and completeness on to-be-perceived objects. A similar notion is 
implicit in the work of consistency theories (Abelson, Aronson, McGuire, Newcomb, Rosenberg & 
Tannenbaum, 1968). One reason for this is that human beings prefer organization. We tend to 
impose structure so that we can grasp the meaning behind physical objects or abstract concepts 
(Goetz, Alexander & Ash, 1992). 

Tne assumption behind the present paper is that teachers should adopt the opposite strategy of 
simplification by integration which captures the inherent tendency for the integration of the objective 
world into subjective wholes. Simplification by integration, therefore, is synonymous with building a 
holistic picture or theme. Once the whole is in sight, the job of learning becomes easier, In China 
there is a well-known saying which goes: "once the net is in control, the openings in the net are 
clear." The message behind this saying is that however complicated a situation may be, if we grasp 
the theme, then everything becomes manageable. Similarly, any realm of human knowledge may be 
packed with detailed facts and concepts. We will be overwhelmed by the amount of information our 
mind needs to Process if we get caught up in endless details. By integrating the information in an 
organized way and concentrating on themes rather than isolated facts and concepts, we can greatly 
increase efficiency. 

An important characteristic in the simplification by integration approach is its emphasis on the 
whole. This is where the assumption of simplification by integration and the whole-theme approach 
meet. If simplification by integration results from an inevitable need in human cognition, the whole- 
theme approach provides a solution to the problem created by that need. 
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An effective way of establishing the theme of a course is to use thematic organizers (Iran- 
Nejad, 1992). A thematic organizer can be used to portray a holistic picture of the entire domcm 
covered by the course. This organizer should be presented at the beginning of the course. Other 
thematic organizers should follow throughout the course to protray the major subthemes. Ushg the 
language of gestalt psychology, a thematic organizer can provide the ground against which the figures 
or various course concepts can be presented. The constant interaction between the ground and the 
figures constitutes an essential part of the ongoing process of learning. 

Another important characteristic of the whole-theme approach is that it views learning as 
reorganization of the learner's own intuitive knowledge and advocates that learning of the new domain 
should occur in the real-world-rich context of this intuitive knowledge base (1KB). Through constant 
reorganization, the learner's 1KB will evolve and grow into a professional knowledge base (PKB) 
(Iran-Nejad, 1992). The importance of the learner's real world experience in the learning process 
has been stressed in other approaches such as contextualism (Jenkins, 1974), situated cognition 
(Brown, Collins & Duguid, 1989), and immersion (Prawat, 1991). Utilizing students' 1KB can 
stimulate their interest and motivation and turn learning into a meaningful and enjoyable experience 
(Iran-Nejad & Cecil, 1992). 

An important strength of the whole-theme approach is that it is compatible with the 
functioning of the brain. According to the biofunctional model (Iran-Nejad, 1989), the human 
nervous system engages in two kinds of activities concurrently: (1) ongoing activity of the brain 
subsystems working in concert (OBA), and (2) momertary firing of microsystems constellations 
(MCF). The OBA provides a stable ongoing context v/hile the MCF emits a series of changing 
figures which stand out against the ground. These two different kinds of brain activity produce 
different types of knowledge. OBA creates the thematic knowledge which is all-involving, 
synthesizing, and domain-comprehensive. MCF produces categorical knowledge which is specific 

6 

o 

ERIC 



Teaching Tests and Measurement 

6 

rather than general, The interaction of OBA and MCF creates a srhema-of-the-moment with both a 
stable (ground) and an ever-changing sequence of figures. Thus learning in the biofunctional model is 
a dynamic life-long process of continuous interaction between OBA and MCF (Iran-Nejad, Marsh 
Clements, 1992). 

In the biofunctional model, learning is no! a mechanical process. Three sources of control 
regulate the organic, as opposed to mechanical, interaction among various learning processes: 
external, active, and dynamic (active and dynamic are also classified as internal). This is in contrast 
with behaviorism in which the learner's performance is regarded as a result of external stimuli. 
Related to external regulation is extrinsic motivation which often characterizes students who study 
under the influence of what Bereiter (1990) called a school work module rather than their own 
interest. The three-source self-regulation theory may also be contrasted with Bereiter's second 
learning module that is common in academic settings where we often find the predominance of 
intentional (or active) learning at the expense of incidental learning (Iran-Nejad, 1990, Bigge, 1982). 
In biofunctional cognition, it is assumed that predominance of active self-regulation pushes learning 
processes toward MCF, categorical knowledge, and sequential learning. The problem with external 
and active regulation is that there is little room left for the spontaneous internal self-regulation 
processes of the learner to engage unintentional and executive-independent or stimulus-independent 
knowledge construction (Iran-Nejad, 1990). A solution to the problem comes through dynamic 
regulation. According to the biofunctional theory, dynamic self regulation is the primary source of 
regulation which results from the spontaneous coordination of brain subsystems (Iran-Nejad & 
Chissom, 1992). In dynamic self-regulation interest is the cause and consequence of learning and 
intrinsic motivation is the principle driving force. When interest and motivation are at work, brain 
subsystems are activated and involved and learning occurs in a multisource fashion (Iran-Nejad, 
Mckeachie & Berliner, 1990). The biofunctional model maintains that learning always involves 
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dynamic self regulation even though one or both of the other two kinds of self-regulation have an 
important role to play in learning. Dynamic self regulation is essential because it supports OBA and 
the creation of domain-comprehensive thematic knowledge. 

An understanding of brain activities, the process of knowledge creation, and sources of 
control have important implications for various aspects of education. For example, the dynamic 
process for professional enrichment and the intrinsic motivation involved in the process best explain 
adult continuing education. The biofunctional model also suggests that the traditional emphasis on 
external and active self regulation and the development of categorical knowledge in the learner are not 
sufficient. Without acquiring the thematic knowledge, learning is fragmented and incomplete. If we 
want our students to acquire new information, we need to assist them to develop both categorical 
knowledge through MCF and thematic knowledge through OBA. This requires us to activate 
spontaneous dynamic self regulation in the learner. Encouraging creativity and utilizing the students' 
rich 1KB seem to be effective ways of achieving this goal. 

Towards Whole-Theme Teaching in Tests and Measurement 

Having realized the limitations of the piecemeal approach, we decided to try the thematic 
approach to teaching Tests and Measurement, a course taught by the first author of this paper. This 
decision led to an almost spontaneous shift from teaching definitions, concepts, and facts to a 
universal focus on domain-comprehensive knowledge of the field of tests and measurement. The 
following discussion describes the major aspects of this shift in focus. 
Use of Thematic Organizers 

Based on her knowledge of the whole-theme approach, her professional experience, and her 
understanding of the course material, the first author produced two thematic organizers, one for the 
entire course and one for the section on classroom statistics (see Figures 1 and 2). From the course 
description and the objectives, the students got some idea about what would be covered in the course. 
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What they did not know was how these topics were related to one another to comprise the domain of 
tests and measurement. We tried to provide that information by using two thematic organizers. 



Insert Figures I and 2 about Here 



One advantage of developing a thematic organizer is that it forces the instructor to think 
systematically about the relationships among various topics covered in a course. How can a teacher 
who teaches a course to students not already have a firm grasp of this? The truth of the matter is that 
more often than not this is indeed the case. What is covered in a course is usually predetermined by 
the established curriculum, if not by the department. So a significant number of instructors end up 
planning for student internalization of curriculum-identified knowledge, locating an external source 
that offers a reasonable match with this curriculum-identified content, and then reproducing that 
content to students in its original external organization. In other words, instructurs do not have to 
fully integrate the topics of a course before they teach it. How well do most instructors know the 
material of the course they teach, particularly the interrelationship among various topics that constitute 
the relevant domain-comprehensive knowledge? How effectively can they communicate it to their 
students in the form of verbal information? Our experience with thematic organizers shows that, to a 
large extent, we can evaluate an instructor's professional expertise by asking him/her to develop a 
thematic organizer for what he/she is teaching. We have learned that in the process of developing a 
thematic organizer a teacher really needs to think carefully about the course material. For example, 
the first author developed the thematic organizers used in this paper. Her first version oi the thematic 
organizer for classroom statistics included three major topics (item analysis, descriptive statistics, and 
psychometric properties). She was initially content with this product because, up to that point, the 
classroom statistics section had not taken any integrated shape at all. However, she soon realized that 
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the organizer needed to be revised. Both item analysis and psychometric properties were related to 
tests. Thus she combined the two together and put them under the heading of test evaluation. 
Descriptive statistics in this class was mainly concerned with score manipulation and interpretation so 
she changed the name for that component to score interpretation, P value, D value, item to total 
correlation, reliability, and validity were related to test evaluation, Central tendency, variability, 
graphic representation of a distribution, standard score transformation, and percentile rank were 
related to score interpretation. The standard error of measurement was related to both components. 
Thus developing a thematic organizer really reshaped and hopefully sharpened the instructor's 
understanding of the course material. More importantly, her students benefitted a lot from it, With 
the assistance of the thematic organizers, the students had a much better chance of understanding the 
theme of the course and how various components were related. 

The construction of a thematic organizer is a continuous process. Due to the constant 
improvement in the instructor's understanding of the course material and changes in course 
requirements, a thematic organizer should always be revised and updated for tighter integration. 
Acknowledging this fact, we will contiue to work on the thematic organizers for our tests and 
measurement course to make them more thematically coherent. 
Students' Intuitive Knowledge Base 

According to the whole-theme approach, learning ought to take place in the rich context of the 
student's own intuitive knowledge base (1KB). We viewed the thematic organizer as an important 
vehicle for harnessing this critical resource. With the aid of the thematic organizer, we hoped to set 
in motion the students' domain-comprehensive 1KB and assist the students to develop it into the 
domain-relevant professional knowledge base (PKB) that was the goal of the course. 

In introductory courses such as tests and measurement, the assumption is often made that no 
domain-specific professional knowledge means no domain-relevant knowledge at all. Consequently, 
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the teaching of many introductory courses begins with a zero-domain-relevant-knowledge assumption. 
We intentionally shifted the focus of our course away from this limiting assumption and toward the 
rich domain-comprehensive-IKB assumption. Naive students all know from real-life experience what 
good performance is, that one's performance is meaningful relative to the performance of others, that 
performance varies from one individual to another, and so on, These are examples of domain- 
comprehensive knowledge that could be reorganized into domain-specific knowledge of tests and 
measurement. For example, students have had ample experience comparing themselves with their 
peers in some performance or other even though they may not have had the opportunity to learn the 
corresponding technical terms they encounter in a tests and measurement course. Thus, deep down in 
their 1KB, students have foundational knowledge about core testing and measurement concepts. With 
this realization, we saw it as our most immediate goal for teaching the course to try to set in holistic 
motion as much of this knowledge as possible, with the aid of thematic organizers and otherwise. We 
saw such focus on the student's domain-comprehensive 1KB as an effective way to increase for them 
the likelihood of seeing the relevance of the course to their daily life, and of making learning easier, 
more meaningful, and possibly more fun to them. 

We tried to give the students in our Tests and Measurement class ample opportunity to discuss 
their relevant experiences, knowledge, and discoveries in the context of the overall thematic organizer 
before introducing formal theories and concepts of the course. Such thematically organized discussion 
is much more focused, easier to guide, and, consequently, much more rewarding for students and 
teachers. Thus, students utilized their prior knowledge much more readily and, hopefully, 
reorganized their 1KB as they shifted gradually toward the professional knowledge base of the course, 

To facilitate this further, students were asked, as part of the regular course requirements, to 
develop assessment material for their future students in the subject matter and grade-level of their 
choice. This involved constructing objectives, a table of specifications, a paper and pencil test, a 
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performance measure, and a grading model. This gave the students further opportunity to apply their 
thematically-reorganized 1KB. In short, thematic organizers, students' intuitive knowledge base, and 
authentic domain-relevant projects are aspects of the whole-theme approach, with thematic organizers 
serving as an important unifying tool. 
Thematic Plus Systematic Teaching 

The guiding principle behind the thematic approach to teaching Tests and Measurement was 
that without knowing the theme students could not organize their own domain-comprehensive 
knowledge on their way towards domain-specific professional knowledge. Once the theme was 
established and maintained throughout the course with the help of our thematic organizers, the stage 
was set for systematic teaching of the course material. We believe this was necessary, particularly for 
the section on classroom statistics where students had a lot of anxiety over math concepts, Statistical 
concepts were usually presented in the following sequence: definition- > formula- > computation -> 
examples- > application. We also planned the class in such a way so that students would have time to 
understand and internalize the course material. This method ensured that students (a) obtained the 
thematic knowledge about the entire domain, (b) learned where each concept fit in the general 
structure of the domain, and (c) developed a thorough understanding of the individual topics covered 
in the domain. We feel that these three objectives can be achieved most readily through the thematic- 
systematic approach taken here. 

We did not have time during this first-time-around application of the whole-theme approach to 
formally evaluate its effectiveness. However, informal observations suggested that the studems 
maintained a positive attitude toward the approach. They found thematic organizers helpful and often 
expressed interest in delving into the thematic knowledge it portrayed. One student wrote: "It is a 
pity many professors do not [use this way of teaching]." Many of the students felt confident about 
their understanding of the course material and their ability to conduct test evaluation and score 
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interpretation in the capacity of a classroom teacher. When we look back at the anxiety with which 
some of the students started the course, particularly over classroom statistics, we find these 
psychological and academic acheivements on the part of our students rewarding. 

Conclusion 

Two common problems in the traditional classroom are (1) knowledge is usually presented in 
a piecemeal fashion, and (2) students* domain-comprehensive intuitive knowledge is often neglected in 
the teaching/learning process. An alternative to the traditional piecemeal approach is the thematic 
approach which adopts the strategy of simplification by integration and strives for the acquisition of 
thematic knowledge. The theoretical basis for this approach is the biofunctional model which 
emphasizes the importance of dynamic self-regulation and advocates the multimodal, intrinsically 
motivated learning approach (Iran-Nejad, 1990, Iran-Nejad & Chissom, 1992; Iran-Nejad, Mckeachie 
& Berliner, 1990). 

The application of the thematic approach to teachhg Tests and Measurement revealed that, 
despite the fact that the piecemeal approach has been the dominating instructional technique in the 
traditional classroom, students had a strong desire to work with their own domain-comprehensive 
knowledge in order to reorganize it into domain-specific knowledge of tests and measurement. Their 
positive response to the thematic organizers used in the course and their willingness and confidence to 
apply the knowledge in their future careers was an indication of the positive effects of the thematic 
approach to teaching the course. 

Adopting the thematic approach is a challenging task both for the teacher and for the student. 
It requires the teacher to shift from the traditional teacher-centered instruction to student-oriented 
coaching in which the learners' 1KB is valued and their unique and creative thinking encouraged. It 
also demands more input from the student. The teacher's function in this learning process should be 
that of a facilitator, showing the student the right direction and providing assistance when necessary. 
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Future research should be directed toward designing the appropriate activities to tap students* rich 
intuitive knowledge base and bridge the gap between academic learning and real life experience. In 
todays classrooms, fragmented knowledge abounds. The present study echoes the outcry of the 
students for whole-theme practice in teaching and learning. This is where a breakthrough can be 
expected in educational reform. This is where we should make changes to improve the quality of 
teaching. 
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